Four statistical frameworks for assessing an immune correlate of protection (surrogate endpoint) from a randomized, controlled, vaccine efficacy trial.

A central goal of vaccine research is to characterize and validate immune correlates of protection (CoPs). In addition to helping elucidate immunological mechanisms, a CoP can serve as a valid surrogate endpoint for an infectious disease clinical outcome and thus qualifies as a primary endpoint for vaccine authorization or approval without requiring resource-intensive randomized, controlled phase 3 trials. Yet, it is challenging to persuasively validate a CoP, because a prognostic immune marker can fail as a reliable basis for predicting/inferring the level of vaccine efficacy against a clinical outcome, and because the statistical analysis of phase 3 trials only has limited capacity to disentangle association from cause. Moreover, the multitude of statistical methods garnered for CoP evaluation in phase 3 trials renders the comparison, interpretation, and synthesis of CoP results challenging. Toward promoting broader harmonization and standardization of CoP evaluation, this article summarizes four complementary statistical frameworks for evaluating CoPs in a phase 3 trial, focusing on the frameworks' distinct scientific objectives as measured and communicated by distinct causal vaccine efficacy parameters. Advantages and disadvantages of the frameworks are considered, dependent on phase 3 trial context, and perspectives are offered on how the frameworks can be applied and their results synthesized.

Durability of Protection Against COVID-19 Through the Delta Surge for the NVX-CoV2373 Vaccine.

BACKGROUND: Protein-based vaccines for COVID-19 provide a traditional vaccine platform with long-lasting protection for non-SARS-CoV-2 pathogens and may complement messenger RNA vaccines as a booster dose. While NVX-CoV2373 showed substantial early efficacy, the durability of protection has not been delineated. METHODS: The PREVENT-19 vaccine trial employed a blinded crossover design; the original placebo arm received NVX-CoV2373 after efficacy was established. Using novel statistical methods that integrate surveillance data of circulating strains with post-crossover cases, we estimated placebo-controlled vaccine efficacy and durability of NVX-CoV2373 against both pre-Delta and Delta strains of SARS-CoV-2. RESULTS: Vaccine efficacy against pre-Delta strains of COVID-19 was 89% (95% CI: 75%, 95%) and 87% (72%, 94%) at 0 and 90 days after 2 doses of NVX-CoV2373, respectively, with no evidence of waning (p=0.93). Vaccine efficacy against the Delta strain was 88% (71%, 95%), 82% (56%, 92%), and 77% (44%, 90%) at 40, 120, and 180 days, respectively, with evidence of waning (p<0.01). In sensitivity analyses, the estimated Delta vaccine efficacy at 120 days ranged from 66% (15%, 86%) to 89% (74%, 95%) per various assumptions of the surveillance data. CONCLUSION: NVX-CoV2373 has high initial efficacy against pre-Delta and Delta strains of COVID-19 with little evidence of waning for pre-Delta strains through 90 days and moderate waning against Delta strains over 180 days.

A controlled effects approach to assessing immune correlates of protection.

An immune correlate of risk (CoR) is an immunologic biomarker in vaccine recipients associated with an infectious disease clinical endpoint. An immune correlate of protection (CoP) is a CoR that can be used to reliably predict vaccine efficacy (VE) against the clinical endpoint and hence is accepted as a surrogate endpoint that can be used for accelerated approval or guide use of vaccines. In randomized, placebo-controlled trials, CoR analysis is limited by not assessing a causal vaccine effect. To address this limitation, we construct the controlled risk curve of a biomarker, which provides the causal risk of an endpoint if all participants are assigned vaccine and the biomarker is set to different levels. Furthermore, we propose a causal CoP analysis based on controlled effects, where for the important special case that the biomarker is constant in the placebo arm, we study the controlled vaccine efficacy curve that contrasts the controlled risk curve with placebo arm risk. We provide identification conditions and formulae that account for right censoring of the clinical endpoint and two-phase sampling of the biomarker, and consider G-computation estimation and inference under a semiparametric model such as the Cox model. We add modular approaches to sensitivity analysis that quantify robustness of CoP evidence to unmeasured confounding. We provide an application to two phase 3 trials of a dengue vaccine indicating that controlled risk of dengue strongly varies with 50$\%$ neutralizing antibody titer. Our work introduces controlled effects causal mediation analysis to immune CoP evaluation.

Optimizing Deep Learning-Based Segmentation of Densely Packed Cells using Cell Surface Markers.

Background: Spatial molecular profiling depends on accurate cell segmentation. Identification and quantitation of individual cells in dense tissues, e.g. highly inflamed tissue caused by viral infection or immune reaction, remains a challenge. Methods: We first assess the performance of 18 deep learning-based cell segmentation models, either pre-trained or trained by us using two public image sets, on a set of immunofluorescence images stained with immune cell surface markers in skin tissue obtained during human herpes simplex virus (HSV) infection. We then further train eight of these models using up to 10,000+ training instances from the current image set. Finally, we seek to improve performance by tuning parameters of the most successful method from the previous step. Results: The best model before fine-tuning achieves a mean Average Precision (mAP) of 0.516. Prediction performance improves substantially after training. The best model is the cyto model from Cellpose. After training, it achieves an mAP of 0.694; with further parameter tuning, the mAP reaches 0.711. Conclusion: Selecting the best model among the existing approaches and further training the model with images of interest produce the most gain in prediction performance. The performance of the resulting model compares favorably to human performance. The imperfection of the final model performance can be attributed to the moderate signal-to-noise ratio i the imageset.

Stochastic Interventional Vaccine Efficacy and Principal Surrogate Analyses of Antibody Markers as Correlates of Protection against Symptomatic COVID-19 in the COVE mRNA-1273 Trial.

The COVE trial randomized participants to receive two doses of mRNA-1273 vaccine or placebo on Days 1 and 29 (D1, D29). Anti-SARS-CoV-2 Spike IgG binding antibodies (bAbs), anti-receptor binding domain IgG bAbs, 50% inhibitory dilution neutralizing antibody (nAb) titers, and 80% inhibitory dilution nAb titers were measured at D29 and D57. We assessed these markers as correlates of protection (CoPs) against COVID-19 using stochastic interventional vaccine efficacy (SVE) analysis and principal surrogate (PS) analysis, frameworks not used in our previous COVE immune correlates analyses. By SVE analysis, hypothetical shifts of the D57 Spike IgG distribution from a geometric mean concentration (GMC) of 2737 binding antibody units (BAU)/mL (estimated vaccine efficacy (VE): 92.9% (95% CI: 91.7%, 93.9%)) to 274 BAU/mL or to 27,368 BAU/mL resulted in an overall estimated VE of 84.2% (79.0%, 88.1%) and 97.6% (97.4%, 97.7%), respectively. By binary marker PS analysis of Low and High subgroups (cut-point: 2094 BAU/mL), the ignorance interval (IGI) and estimated uncertainty interval (EUI) for VE were [85%, 90%] and (78%, 93%) for Low compared to [95%, 96%] and (92%, 97%) for High. By continuous marker PS analysis, the IGI and 95% EUI for VE at the 2.5th percentile (519.4 BAU/mL) vs. at the 97.5th percentile (9262.9 BAU/mL) of D57 Spike IgG concentration were [92.6%, 93.4%] and (89.2%, 95.7%) vs. [94.3%, 94.6%] and (89.7%, 97.0%). Results were similar for other D29 and D57 markers. Thus, the SVE and PS analyses additionally support all four markers at both time points as CoPs.

Examining protective effects of SARS-CoV-2 neutralizing antibodies after vaccination or monoclonal antibody administration.

While new vaccines for SARS-CoV-2 are authorized based on neutralizing antibody (nAb) titer against emerging variants of concern, an analogous pathway does not exist for preventative monoclonal antibodies. In this work, nAb titers were assessed as correlates of protection against COVID-19 in the casirivimab + imdevimab monoclonal antibody (mAb) prevention trial (ClinicalTrials.gov #NCT4452318) and in the mRNA-1273 vaccine trial (ClinicalTrials.gov #NCT04470427). In the mAb trial, protective efficacy of 92% (95% confidence interval (CI): 84%, 98%) is associated with a nAb titer of 1000 IU50/ml, with lower efficacy at lower nAb titers. In the vaccine trial, protective efficacies of 93% [95% CI: 91%, 95%] and 97% (95% CI: 95%, 98%) are associated with nAb titers of 100 and 1000 IU50/ml, respectively. These data quantitate a nAb titer correlate of protection for mAbs benchmarked alongside vaccine induced nAb titers and support nAb titer as a surrogate endpoint for authorizing new mAbs.

Comparing antibody assays as correlates of protection against COVID-19 in the COVE mRNA-1273 vaccine efficacy trial.

The best assay or marker to define mRNA-1273 vaccine-induced antibodies as a correlate of protection (CoP) is unclear. In the COVE trial, participants received two doses of the mRNA-1273 COVID-19 vaccine or placebo. We previously assessed IgG binding antibodies to the spike protein (spike IgG) or receptor binding domain (RBD IgG) and pseudovirus neutralizing antibody 50 or 80% inhibitory dilution titer measured on day 29 or day 57, as correlates of risk (CoRs) and CoPs against symptomatic COVID-19 over 4 months after dose. Here, we assessed a new marker, live virus 50% microneutralization titer (LV-MN50), and compared and combined markers in multivariable analyses. LV-MN50 was an inverse CoR, with a hazard ratio of 0.39 (95% confidence interval, 0.19 to 0.83) at day 29 and 0.51 (95% confidence interval, 0.25 to 1.04) at day 57 per 10-fold increase. In multivariable analyses, pseudovirus neutralization titers and anti-spike binding antibodies performed best as CoRs; combining antibody markers did not improve correlates. Pseudovirus neutralization titer was the strongest independent correlate in a multivariable model. Overall, these results supported pseudovirus neutralizing and binding antibody assays as CoRs and CoPs, with the live virus assay as a weaker correlate in this sample set. Day 29 markers performed as well as day 57 markers as CoPs, which could accelerate immunogenicity and immunobridging studies.

Immune correlates analysis of a phase 3 trial of the AZD1222 (ChAdOx1 nCoV-19) vaccine.

In the phase 3 trial of the AZD1222 (ChAdOx1 nCoV-19) vaccine conducted in the U.S., Chile, and Peru, anti-spike binding IgG concentration (spike IgG) and pseudovirus 50% neutralizing antibody titer (nAb ID50) measured four weeks after two doses were assessed as correlates of risk and protection against PCR-confirmed symptomatic SARS-CoV-2 infection (COVID-19). These analyses of SARS-CoV-2 negative participants were based on case-cohort sampling of vaccine recipients (33 COVID-19 cases by 4 months post dose two, 463 non-cases). The adjusted hazard ratio of COVID-19 was 0.32 (95% CI: 0.14, 0.76) per 10-fold increase in spike IgG concentration and 0.28 (0.10, 0.77) per 10-fold increase in nAb ID50 titer. At nAb ID50 below the limit of detection (< 2.612 IU50/ml), 10, 100, and 270 IU50/ml, vaccine efficacy was -5.8% (-651%, 75.6%), 64.9% (56.4%, 86.9%), 90.0% (55.8%, 97.6%) and 94.2% (69.4%, 99.1%). These findings provide further evidence towards defining an immune marker correlate of protection to help guide regulatory/approval decisions for COVID-19 vaccines.

Antibody Correlates of Protection From Severe Respiratory Syncytial Virus Disease in a Vaccine Efficacy Trial.

Background: Respiratory syncytial virus (RSV) can cause serious lung infections in young children and there is currently no available vaccine. Methods: We used complementary statistical frameworks to analyze 4 RSV serology measurements in mothers and their infants in South Africa who participated in a phase 3 maternal immunization trial of an RSV F protein nanoparticle vaccine as correlates of risk and of protection against different RSV disease endpoints. Results: We found evidence to support each antibody measurement-encompassing RSV-neutralizing antibodies and F surface glycoprotein-binding antibodies-as an inverse correlate of risk of RSV-associated acute lower respiratory tract infection with severe hypoxia in at least 1 framework, with vaccine-induced fold-rise from the maternal enrollment to day 14 samples of anti-F immunoglobulin G (IgG) binding antibodies having the most consistent evidence. This evidence includes a significant association of fold-rise anti-F IgG with vaccine efficacy (VE); achieving a baseline covariate-adjusted VE of 75% requires a vaccine-induced maternal anti-F IgG fold-rise of around 16. Neither multivariable logistic regression nor superlearning analyses showed benefit to including multiple time points or assays in the same model, suggesting a parsimonious correlate. Post hoc exploratory analyses supported adherence of vaccine-induced maternal anti-F IgG fold-rise to the Prentice criteria for a valid surrogate endpoint. Conclusions: Our results suggest that the vaccine induced protective anti-F antibody responses. If this finding is confirmed, VE could potentially be augmented by increasing these responses.

Correlates of Protection, Thresholds of Protection, and Immunobridging among Persons with SARS-CoV-2 Infection.

Several studies have shown that neutralizing antibody levels correlate with immune protection from COVID-19 and have estimated the relationship between neutralizing antibodies and protection. However, results of these studies vary in terms of estimates of the level of neutralizing antibodies required for protection. By normalizing antibody titers, we found that study results converge on a consistent relationship between antibody levels and protection from COVID-19. This finding can be useful for planning future vaccine use, determining population immunity, and reducing the global effects of the COVID-19 pandemic.

Immune correlates analysis of the PREVENT-19 COVID-19 vaccine efficacy clinical trial.

In the PREVENT-19 phase 3 trial of the NVX-CoV2373 vaccine (NCT04611802), anti-spike binding IgG concentration (spike IgG), anti-RBD binding IgG concentration (RBD IgG), and pseudovirus 50% neutralizing antibody titer (nAb ID50) measured two weeks post-dose two are assessed as correlates of risk and as correlates of protection against COVID-19. Analyses are conducted in the U.S. cohort of baseline SARS-CoV-2 negative per-protocol participants using a case-cohort design that measures the markers from all 12 vaccine recipient breakthrough COVID-19 cases starting 7 days post antibody measurement and from 639 vaccine recipient non-cases. All markers are inversely associated with COVID-19 risk and directly associated with vaccine efficacy. In vaccine recipients with nAb ID50 titers of 50, 100, and 7230 international units (IU50)/ml, vaccine efficacy estimates are 75.7% (49.8%, 93.2%), 81.7% (66.3%, 93.2%), and 96.8% (88.3%, 99.3%). The results support potential cross-vaccine platform applications of these markers for guiding decisions about vaccine approval and use.

Respiratory syncytial virus prevention within reach: the vaccine and monoclonal antibody landscape.

Respiratory syncytial virus is the second most common cause of infant mortality and a major cause of morbidity and mortality in older adults (aged >60 years). Efforts to develop a respiratory syncytial virus vaccine or immunoprophylaxis remain highly active. 33 respiratory syncytial virus prevention candidates are in clinical development using six different approaches: recombinant vector, subunit, particle-based, live attenuated, chimeric, and nucleic acid vaccines; and monoclonal antibodies. Nine candidates are in phase 3 clinical trials. Understanding the epitopes targeted by highly neutralising antibodies has resulted in a shift from empirical to rational and structure-based vaccine and monoclonal antibody design. An extended half-life monoclonal antibody for all infants is likely to be within 1 year of regulatory approval (from August, 2022) for high-income countries. Live-attenuated vaccines are in development for older infants (aged >6 months). Subunit vaccines are in late-stage trials for pregnant women to protect infants, whereas vector, subunit, and nucleic acid approaches are being developed for older adults. Urgent next steps include ensuring access and affordability of a respiratory syncytial virus vaccine globally. This review gives an overview of respiratory syncytial virus vaccines and monoclonal antibodies in clinical development highlighting different target populations, antigens, and trial results.

Immune correlates analysis of the ENSEMBLE single Ad26.COV2.S dose vaccine efficacy clinical trial.

Measuring immune correlates of disease acquisition and protection in the context of a clinical trial is a prerequisite for improved vaccine design. We analysed binding and neutralizing antibody measurements 4 weeks post vaccination as correlates of risk of moderate to severe-critical COVID-19 through 83 d post vaccination in the phase 3, double-blind placebo-controlled phase of ENSEMBLE, an international randomized efficacy trial of a single dose of Ad26.COV2.S. We also evaluated correlates of protection in the trial cohort. Of the three antibody immune markers we measured, we found most support for 50% inhibitory dilution (ID50) neutralizing antibody titre as a correlate of risk and of protection. The outcome hazard ratio was 0.49 (95% confidence interval 0.29, 0.81; P = 0.006) per 10-fold increase in ID50; vaccine efficacy was 60% (43%, 72%) at non-quantifiable ID50 (<2.7 IU50 ml-1) and increased to 89% (78%, 96%) at ID50 = 96.3 IU50 ml-1. Comparison of the vaccine efficacy by ID50 titre curves for ENSEMBLE-US, the COVE trial of the mRNA-1273 vaccine and the COV002-UK trial of the AZD1222 vaccine supported the ID50 titre as a correlate of protection across trials and vaccine types.

The clinical effect of point-of-care HIV diagnosis in infants: a systematic review and meta-analysis.

BACKGROUND: Timely diagnosis and treatment of HIV is crucial in HIV-exposed infants to prevent the high rates of mortality seen during the first 2 years of life if HIV is untreated. However, challenges with sample transportation, testing, and result delivery to caregivers have led to long delays in treatment initiation. We aimed to compare the clinical effect of point-of-care HIV testing versus laboratory-based testing (standard of care) in HIV-exposed infants. METHODS: We did a systematic review and meta-analysis and searched PubMed, MEDLINE, Cochrane Central Register of Controlled Trials, Embase, Conference Proceedings Citation Index-Science, and WHO Global Index Medicus, from Jan 1, 2014, to Aug 31, 2020. Studies were included if they pertained to the use of point-of-care nucleic acid testing for infant HIV diagnosis, had a laboratory-based nucleic acid test as the comparator or standard of care against the index test (same-day point-of-care testing), evaluated clinical outcomes when point-of-care testing was used, and included HIV-exposed infants aged younger than 2 years. Studies were excluded if they did not use a laboratory-based comparator, a nucleic acid test that had been approved by a stringent regulatory authority, or diagnostic-accuracy or performance evaluations (eg, no clinical outcomes included). Reviews, non-research letters, commentaries, and editorials were also excluded. The risk of bias was evaluated using the ROBINS-I framework. Data were extracted from published reports. Data from all studies were analysed using frequency statistics to describe the overall populations evaluated and their results. Key outcomes were time to result delivery and antiretroviral therapy initiation, and proportion of HIV-positive infants initiated on antiretroviral therapy within 60 days after sample collection. FINDINGS: 164 studies were identified by the search and seven were included in the analysis, comprising 37 377 infants in total across 15 countries, including 25 170 (67%) who had point-of-care HIV testing and 12 207 (33%) who had standard-of-care testing. The certainty of evidence was high. Same-day point-of-care testing led to a significantly shorter time between sample collection and result delivery to caregivers compared with standard-of-care testing (median 0 days [95% CI 0-0] vs 35 days [35-37]). Time from sample collection to antiretroviral therapy initiation in infants found to be HIV-positive was significantly lower with point-of-care testing compared with standard of care (median 0 days [95% CI 0-1] vs 40 days [36-44]). When each study's result was weighted equally, 90·3% (95% CI 76·7-96·5) of HIV-positive infants diagnosed using point-of-care testing had started antiretroviral therapy within 60 days of sample collection, compared with only 51·6% (27·1-75·7) who had standard-of-care testing (odds ratio 8·74 [95% CI 6·6-11·6]; p<0·0001). INTERPRETATION: Overall, the certainty of the evidence in this analysis was rated as high for the primary outcomes related to result delivery and treatment initiation, with no serious risk of bias, inconsistency, indirectness, or imprecision. In HIV-exposed infants, same-day point-of-care HIV testing was associated with significantly improved time to result delivery, time to antiretroviral therapy initiation, and proportion of HIV-positive infants starting antiretroviral therapy within 60 days compared with standard of care. FUNDING: The Bill & Melinda Gates Foundation.

A controlled effects approach to assessing immune correlates of protection.

An immune correlate of risk (CoR) is an immunologic biomarker in vaccine recipients associated with an infectious disease clinical endpoint. An immune correlate of protection (CoP) is a CoR that can be used to reliably predict vaccine efficacy (VE) against the clinical endpoint and hence is accepted as a surrogate endpoint that can be used for accelerated approval or guide use of vaccines. In randomized, placebo-controlled trials, CoR analysis is limited by not assessing a causal vaccine effect. To address this limitation, we construct the controlled risk curve of a biomarker, which provides the causal risk of an endpoint if all participants are assigned vaccine and the biomarker is set to different levels. Furthermore, we propose a causal CoP analysis based on controlled effects, where for the important special case that the biomarker is constant in the placebo arm, we study the controlled vaccine efficacy curve that contrasts the controlled risk curve with placebo arm risk. We provide identification conditions and formulae that account for right censoring of the clinical endpoint and two-phase sampling of the biomarker, and consider G-computation estimation and inference under a semiparametric model such as the Cox model. We add modular approaches to sensitivity analysis that quantify robustness of CoP evidence to unmeasured confounding. We provide an application to two phase 3 trials of a dengue vaccine indicating that controlled risk of dengue strongly varies with 50$\%$ neutralizing antibody titer. Our work introduces controlled effects causal mediation analysis to immune CoP evaluation.

HIV viral load assays when used with whole blood perform well as a diagnostic assay for infants.

OBJECTIVE: Over the past several years, only approximately 50% of HIV-exposed infants received an early infant diagnosis test within the first two months of life. While high attrition and mortality account for some of the shortcomings in identifying HIV-infected infants early and putting them on life-saving treatment, fragmented and challenging laboratory systems are an added barrier. We sought to determine the accuracy of using HIV viral load assays for infant diagnosis of HIV. METHODS: We enrolled 866 Ugandan infants between March-April 2018 for this study after initial laboratory diagnosis. The median age was seven months, while 33% of infants were less than three months of age. Study testing was done using either the Roche or Abbott molecular technologies at the Central Public Health Laboratory. Dried blood spot samples were prepared according to manufacturer-recommended protocols for both the qualitative and quantitative assays. Viral load test samples for the Roche assay were processed using two different buffers: phosphate-buffered saline (PBS: free virus elution viral load protocol [FVE]) and Sample Pre-Extraction Reagent (SPEX: qualitative buffer). Dried blood spot samples were processed for both assays on the Abbott using the manufacturer's standard infant diagnosis protocol. All infants received a qualitative test for clinical management and additional paired quantitative tests. RESULTS: 858 infants were included in the analysis, of which 50% were female. Over 75% of mothers received antiretroviral therapy, while approximately 65% of infants received infant prophylaxis. The Roche SPEX and Abbott technologies had high sensitivity (>95%) and specificity (>98%). The Roche FVE had lower sensitivity (85%) and viral load values. CONCLUSIONS: To simplify and streamline laboratory practices, HIV viral load may be used to diagnose HIV infection in infants, particularly using the Roche SPEX and Abbott technologies.

Immune Correlates Analysis of a Single Ad26.COV2.S Dose in the ENSEMBLE COVID-19 Vaccine Efficacy Clinical Trial.

Anti-spike IgG binding antibody, anti-receptor binding domain IgG antibody, and pseudovirus neutralizing antibody measurements four weeks post-vaccination were assessed as correlates of risk of moderate to severe-critical COVID-19 outcomes through 83 days post-vaccination and as correlates of protection following a single dose of Ad26.COV2.S COVID-19 vaccine in the placebo-controlled phase of ENSEMBLE, an international, randomized efficacy trial. Each marker had evidence as a correlate of risk and of protection, with strongest evidence for 50% inhibitory dilution (ID50) neutralizing antibody titer. The outcome hazard ratio was 0.49 (95% confidence interval 0.29, 0.81; p=0.006) per 10-fold increase in ID50; vaccine efficacy was 60% (43, 72%) at nonquantifiable ID50 (< 2.7 IU50/ml) and rose to 89% (78, 96%) at ID50 = 96.3 IU50/ml. Comparison of the vaccine efficacy by ID50 titer curves for ENSEMBLE-US, the COVE trial of the mRNA-1273 vaccine, and the COV002-UK trial of the AZD1222 vaccine supported consistency of the ID50 titer correlate of protection across trials and vaccine types.

Testing a global null hypothesis using ensemble machine learning methods.

Testing a global null hypothesis that there are no significant predictors for a binary outcome of interest among a large set of biomarker measurements is an important task in biomedical studies. We seek to improve the power of such testing methods by leveraging ensemble machine learning methods. Ensemble machine learning methods such as random forest, bagging, and adaptive boosting model the relationship between the outcome and the predictor nonparametrically, while stacking combines the strength of multiple learners. We demonstrate the power of the proposed testing methods through Monte Carlo studies and show the use of the methods by applying them to the immunologic biomarkers dataset from the RV144 HIV vaccine efficacy trial.